1. Field of the Invention
The present invention relates to a data storage system for storing a number of versions of data objects, for example program files.
2. Description of the Related Art
In the development of many software products, for example operating systems, database management software and word processors, typically a large number of software developers work upon individual aspects of the software product. These individual aspects are known as objects. The development work is divided amongst the system developers and each, either individually or in groups, works upon particular objects. The objects are brought together to form the final software product for shipping to the consumer.
Such software development produces considerable technical problems in that, as each object is developed individually, the manner in which the objects interact with each other is also affected. It is therefore vital to monitor the development of each object and to retain copies of the objects at various stages of their development.
Configuration management tools have been developed for this purpose. These typically provide a “versioning” facility in which versions of each object are stored as they are developed. Versioning generally produces a new copy of an object each time it is modified and saved by the software developer during their work. One example of such software is ClearCase by Rational. This software manages each of the objects which are stored in a data “repository” that is accessible by each software developer. Whenever a software developer wishes to work on an object, the object file is “checked out” from the repository. This involves the copying of the specific file to a location where it is made available for modification by the developer. Following modification, the new version of the object file is checked back into the repository and is labelled with a new version name.
In large scale projects, there may be thousands of software developers, many of whom regularly produce new versions of the objects.
The structure of the repository is determined by metadata which in turn is arranged according to a corresponding metamodel. Recently, there have been developments away from file based objects with a view to managing more general information for example information relating to the structure and operation of businesses, so called “enterprise” metadata. One example of how such information can be manipulated and stored is by the use of relational tables. Designer 6i, available from Oracle Corporation is capable of handling such objects and also provides a versioning facility. This allows the expression of powerful queries of the metadata whereas it is much more difficult to query this information if it is contained in files.
Some conventional systems also encounter practical volume limitations which seriously affect their scalability. One known method of addressing this problem involves using multiple instances of the repository, each instance being known as a “VOB”. However, in large projects there may be a requirement for many such VOBs and this leads to large administration costs.
A second major problem with conventional systems is that the metadata are actually stored in the files that also contain the objects' data. This leads to performance problems in operations that involve the accessing of large amounts of metadata.
One reason for this is that present repositories such as that provided by Designer 6i, are organised as an “object table” according to a metamodel. Each version of every object is stored in the object table along with a corresponding “primary key” which uniquely identifies the version of the object. The primary key for each row of the object table is generated using a known algorithm. The size of the object table is directly dependent upon the number of objects stored and the number of versions of each object. For large software development projects in particular, the size of the object table generates problems due by the processor time required to address and access the object table contents.
One procedure that is particularly time consuming in known systems is the formation of a “stripe”. This is the downloading of particular versions of a number of objects that are related in some way. This is necessary for analysing the interaction of the objects and indeed in the production of the software product as a whole. In conventional systems the formation of a stripe is treated as a single request from a developer in a similar manner to any other request such as for the modification of an object. However, stripe formation requires a great deal of processor time because a large number of objects are required to be accessed. The storage of the metadata with the object data seriously hinders this process and in addition, because the stripe formation is treated as a single request, it is required to be completed before any further tasks may be performed upon the objects in question. For this reason no access will be granted to other users of the particular objects accessed during the stripe formation, until this process has been completed.
Many hours of system developer time may be lost as conventionally a single typical stripe download may take a number of hours to process. In systems with high administration costs, this lost time is particularly significant.
Known systems are also provided with a “locking mechanism” which is used to ensure that updates and access of the repository occurs in the correct order. This prevents software developers from accessing any objects which are currently being accessed by another process. For a large table this may cause significant delays. The formation of a stripe is particularly costly in terms of developer time as, because the stripe formation is treated as a single request, the locking mechanism operates to prevent access to any of the objects involved until the stripe formation process has been completed.
The problems identified above are becoming increasingly significant in that software is ever more complex and the structure of known configuration management systems is becoming increasingly unsuitable for managing their development.